Head mounted displays are widely used for three-dimensional (3D) visualizations tasks such as simulators, surgery planning, medical training, and engineering design. Traditionally the head mounted display technology has been based on eyepiece optics. Issues associated with an eyepiece-based system include lack of compactness and large distortion for wide field of view designs, because the aperture stop of the system is located outside of the lens which has promoted other designs such as the head-mounted projection displays.
Head mounted projection display is a technology that is positioned at the boundary between conventional head mounted displays and projection displays such as the computer-automated virtual environment (CAVE). U.S. Pat. No. 5,621,572 issued to Fergason on Apr. 15, 1997 describes an optical system for head mounted display using retroreflectors and discloses a method of displaying an image. Other head mounted projection displays include R. Kijima and T. Ojika, Transition between virtual environment and workstation environment with projective head-mounted display, Proceeding of the IEEE 1997 Virtual Reality Annual International Symposium, Los Alamitos, Calif., pp. 130-137 and C. Cruz-Neira et al., Surround-screen projection-based virtual reality: the design and implementation of the CAVE, Conference of Computer Graphics, Anaheim, Calif. (1993), Proc. Of ACM SIGGRAPH 93, pp. 135-142.
A head mounted projection display (HMPD) consisting of a pair of miniature projections lenses, beam splitters, miniature displays mounted on the helmet, and a flexible, non distorting retro-reflective sheeting material strategically placed in the environment is described in U.S. Pat. No. 5,572,229 issued to Fisher on Nov. 5, 1996 which discloses a head-mounted projection display system using a beam splitter and method of making same. K. F. Arrington, and G. A. Geri, “Conjugate-Optical Retroreflector Display System: Optical Principles and Perceptual Issues,” Journal of the SID, August 2000, pp. 9-10 describes systems wherein an image on the micro-display is projected through the lens onto the material, and then it is retro-reflected back to the entrance pupil of the eye, which is conjugate to the exit pupil of the optics through the beam splitter.
The HMPD technology has a few distinguishing advantages over conventional eyepiece HMDs as described in H. Hua, Y. Ha, and J. P. Rolland, Design of an ultralight and compact projection lens, Applied Optics, Vol. 42, No. 1, (January 2003). Along with the see-through capability which allows optical augmentation of the real world (augmented reality), the HMPD also provides correct occlusion of computer generated content by real objects. A real object placed between the beam splitter and the retro-reflective sheeting will effectively block rays thus providing occlusion of the virtual image. Because of its flexibility, the retroreflective material can be applied anywhere in the physical space and can be tailored to arbitrary shapes without introducing additional distortion. Compared to conventional eyepiece-based see-through HMDs, utilization of projection optics allows for reduced optical distortion across similar fields of view and an increase in the field of view without sacrificing compactness, since the size of the optics does not scale with field of view.
While head-mounted display technologies have undergone significant developments in the last decade, they have suffered from tradeoffs and limitations in capability, which impose critical effects on visualization accuracy and user performance. Among the tradeoffs and limitations, the ignorance of eye movement is often an overlooked aspect. The functional benefits of an integrated HMPD with eye tracking capability solution for human-computer, multi-modal interfaces, and gaze-contingent foveated displays have been recognized, but very few efforts have been made towards a low-level integration.